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DETAILED ACTION 

1 . Claims 1-27 and 58-65 are pending in this application and claims 66-67 are 
newly added on 09/02/2005. 

Response to Amendment 

2. The DECLARATION filed on 09/02/005 under 37 CFR 1 .131 has been 
considered but is ineffective to overcome the Shneiderman reference. 

A. Upon the reviewing the DECLARATION OF ELIZABETH SHARPE AND 
JOHN FRASER and comparing the records shown in the Exhibit B, the period between 
August 3 and September 21 is not accounted for. 

B. Exhibit A only discloses the concept of the claims 1 -27; however, claims 
58-65 discloses the different concept including "the authenticating an operator as a 
member of the group of user, identifying candidate identification values based upon the 
group with whom the operator is authenticated..." Exhibit A discloses a basic inventive 
concept, which do not including these limitations. 

MPEP 715.02, SWEARING BEHIND ONE OF A PLURALITY OF COMBINED 
REFERENCES, applicant's 37 CFR 1.131 affidavit must show possession of either the 
whole invention as claimed or something falling within the claim(s) prior to the affective 
date of the reference being antedated; it is not enough merely to show possession of 
what the reference happens to show if the reference does not teach the basic inventive 
concept. 
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Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-27 and 66 are rejected under 35 U.S.C. 103(a) as being unpatentable 

over Shneiderman et al. (Direct Annotation: A Drag-and-Drop Strategy for Labeling 

Photos, July 2000) in view Mizoguchi Yoshiyuki (EP 0 678 816 A2). 

Regarding on claims 1,16 and 17. Shneiderman teaches a method or archiving 
and retrieving digital media items, comprising: 

Receiving a user input identifying a group of users to which an archiving belongs 
(the user log on to the system) (fig. 4); 

Receiving archiving input data identifying: a digital media item to be archived for 
the group the group (add photo) (fig. 4), the user's selection of zero or more group event 
types from a predetermined plurality of group event types specific to the group 
(visualizing personal histories workshop July 1997) (fig. 4), the user's selection of zero 
or more persons in the group (loskowski, Sharon, Li, Jia and Plaisant, Catherine) (fig. 
4), and the user's selection of period (the starting date and the ending date) (fig. 4); 

Generating index information using the received user archiving input (each photo 
should have a unique reference and photos with the same reference are not allowed to 
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the stored in the in this table even through they have different attribute values) (col. 6, 
lines 14-17); 

Repeating the reception of archiving input data, the generation of the index 
information and the storing of the index information for a plurality of digital media items 
(the processing continue for all other picture in my document) (fig. 4) 

Shneiderman does not explicitly teach receiving retrieval input data representing 
a selection of a default or zero or more group event types from the predetermined 
plurality of group event types for the group, a selection of a default or zero or more 
persons in the group, a selection of time period; and using the selections and the 
identified group to retrieve and output digital media items that match the selection. 
However, Mizoguchi discloses "in the display state show in Fig. 9B, when one of the 
"person key", "place" key, and "other data" key in the upper right portion 7b is operated, 
a list of person information, place information, and other information stored as the 
associated data of the image data in the memory card MC are displayed in accordance 
with the operated search key. When given information "X" is operated in the list, images 
having the information "X" as the associated information are searched and displayed 
from the last one" (col. 10, lines 19-28). This suggests the user selecting the list of 
user, place and other data for a query. Therefore, it would have been obvious to one 
ordinary skill in the art at the time of the invention was made to modify Shneiderman's 
system to include user selecting the list of user, place and other data as taught by 
Mizoguchi in order to retrieve media with different parameters. 
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Regarding on claims 2 and 18, Shneiderman teaches a method recited in claim 1 
wherein the retrieval input data comprises a user input from user identifying a group to 
which the user belongs and the digital media items are retrieved using the group 
identified for the other user in the user retrieval input (user log in with the system which 
identify the person in the library) (fig. 4). 

Regarding on claims 3 and 19, Shneiderman teaches a method recited in claim 1 
including defining the distinct groups of people, and defining group event types that are 
appropriate for members of the groups to distinguish episodic events memorable to the 
group (each of the photo being association with group of users in the photo) (fig. 4). 

Regarding on claims 4 and 20, Shneiderman teaches a method recited in claim 1 
including receiving said digital media item to be archived, and storing said digital media 
item in association with the index information (col. 2, lines 14-17). 

Regarding on claims 5 and 21 , Shneiderman teaches the method recited in claim 
1 including receiving archiving input data identifying a digital media item as being 
associated with a memorable high point in the mind of the user (time is the high point of 
the photo) (fig. 4). 

Regarding on claims 6 and 22, Shneiderman teaches the method recited in claim 
5 wherein the retrieval input data includes an input selecting memorable high points 
(searching by names of people in each photo) (col. 3, lines 8-9). 

Regarding on claims 7 and 23, Shneiderman teaches the method recited in claim 
1 wherein the index information includes an identification of a media type of the digital 
media item (col. 3, lines 4-8). 
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Regarding on claims 8 and 24, Shneiderman teaches the method recited in claim 
7 wherein the retrieval input data includes an input identifying a media type, and the 
digital media items are retrieved and output based on the identified media type (col. 1, 
lines 4-8). 

Regarding on claims 9 and 25-26, Shneiderman teaches the method recited the 
method in claim 1 including receiving archiving input data identifying a plurality of digital 
media items and an input identifying the digital media items to be associated as 
perceived by the user, wherein the index information is generated to include the 
identified association (the association with people, month and person) (fig. 6 in page 7). 

Regarding on claim 10, Shneiderman teaches the method recited in claim 9 
wherein when digital media items are retrieved and output as a result of the user 
retrieval input, any digital media items having the identified association in the index 
information are automatically identified for retrieval and output (page. 3, col. 2, lines 8- 
10). 

Regarding on claim 11, Shneiderman teaches the method recited in claim 10 
wherein the automatically identified digital media items are automatically retrieved and 
output (page. 3, col. 2, lines 8-10). 

Regarding on claim 12, Shneiderman teaches a method recited in claim 10 
including outputting a notification to a user that associated digital media items are 
available, and retrieving and outputting automatically identified digital media items in 
response to a user input page. 3, col. 2, lines 8-10). 
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Regarding on claims 13 and 27, Shneiderman teaches the method recited in 
claim 1 further comprising: 

Receiving a user request for automatic nostalgic retrieval (page. 3, col. 2, lines 8- 

10), 

Automatically generating an initial set of said selections (page. 3, col. 2, lines 8- 

10), 

Using the modified selections to retrieve and output digital media items (page. 3, 
col. 2, lines 8-10); and 

Repeating the modifying, and retrieval and output steps (page. 3, col. 2, lines 8- 

10). 

Claims 14-15 are rejected under the same reason as claim 1. 
Regarding on claim 66, Shneiderman discloses method for archiving digital 
media items, comprising: 

Building a database that includes: 

Digital media items to be archived for the social group (fig. 4). and 
Index information for the digital media items, each instance of index 
information created from archiving input data identifying a user's response to a query 
that identifies a plurality of event types previously registered as associated with the 
social group, and persons previously registered as members of the social group (each 
photo should have a unique reference and photos with the same reference are not 
allowed to the stored in the in this table even through they have different attribute 
values) (col. 6, lines 14-17 and fig. 4). Shneiderman does not explicitly teach receiving 
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a user input identifying a social group to which an archiving user belongs. However, 
Mizoguchi discloses receiving a user input identifying a social group to which an 
archiving user belongs (col. 10, lines 19-25). Therefore, it would have been obvious to 
one ordinary skill in the art at the tinne of the invention was nnade to modify 
Shneiderman' system to include retrieving the group of people by inputting one as 
taught by Mizoguchi in order to retrieve media relating to one or all people in the group. 

5. Claim 67 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Mizoguchi Yoshiyuki (EP 0 678 816 A2) in view Shneiderman et al. (Direct Annotation: 
A Drag-and-Drop Strategy for Labeling Photos, July 2000). 

Regarding on claim 67, Mizoguchi teaches a method of searching digital media 
items, comprising: 

Receiving a user input identifying a social group for which a search is to be 
conducted (col. 10, lines 19-25); 

displaying a query that identifies the candidate identification values and including 
valid selections of an event type of the social group, persons from the social group and 
time (col. 10, lines 35-57), 

Responsive to selection criteria made in response to the query, searching a 
database and retrieving digital media items that satisfy the selection criteria (col. 1 1 , 
lines 9-12). Mizoguchi does not explicitly discloses Identifying candidate identification 
values based upon the social group. However, 



Application/Control Number: 09/670,635 Page 9 

Art Unit: 2162 

6. Claims 58-65 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Shneiderman et al. (Direct Annotation: A Drag-and-Drop Strategy for Labeling Photos, 
July 2000) in view Astle (US. Patent No. 5,485,61 1 ). 

Regarding on claim 58, Shneiderman teaches a media archival method, under 
control of an operator who is a member of a group: 

Authenticating an operator as a member of a group of users (the user log in to 
the system to identify the photo in the samplehcilibraryl. Mbb) (fig. 4), 

Identifying candidate identification values based upon the group with whom the 
operator is authenticated (one the user log in the C:\my photos\... the images and 
groups of people are identified) (fig. 4), 

Querying the operator for selection of identification data to be associated with a 
digital media item, the query identifying the candidate identification values and including 
valid selections of an event type and persons from the group and time (after the log 
process successful, the system displays graphical user interface to allow the user to 
select people, date, location and description) (fig. 4). Shneiderman does not explicitly 
teach generating index information from a response of the operator, and storing the 
index information in association with the digital media item. However, Shneiderman 
discloses "each photo should have a unique reference and photos with the same 
references are not allowed to be stored in this table even through they have different 
values" (col. 6, lines 14-17). On the other hand, Astle discloses "an index can be 
prepared by the consumer while viewing the video footage stored within the database. 
The location on the particular video cassette may be denoted by a time index or a 
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counter index, for example. Thus, the user-prepared index may indicate that he hot-air 
balloon event appears on video cassette number 12, starting at time 1 :17:23 (in hours: 
minutes: seconds format) from the beginning of the video cassette, and/or at counter 
number 2351 from the beginning of the tape" (col. 2, lines 1 6-25). This suggests that 
the index is created with event and time. Therefore, it would have been obvious to one 
ordinary skill in the art at the time of the invention was made to modify Shneiderman's 
system to include the creating of an index utilizing the event and time as taught by Astle 
in order to retrieve the image using different parameters. 

Regarding on claims 59 and 63, Shneiderman teaches the archived method 
recited in claim 58, wherein the candidate identification values for persons include 
names of group members (selecting names in the library) (fig. 4). 

Regarding on claims 60 and 64, Shneiderman teaches the archival method 
recited in claim 58, wherein the stored index information include names of group 
members (history visualization workshop) (fig. 4). 

Regarding on claims 61 and 65, Shneiderman teaches the archival method 
recited in claim 58, wherein the stored index information includes a flag that 
distinguishes high point items from other items, and the method further comprises 
setting the flag if the operator response includes an indication that the digital media item 
is a high point (show name labels) (fig. 4). 

Claim 62 is rejected under the same reason as claim 58. 
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Conclusion 

7. THIS ACTION IS MADE FINAL Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Contact Information 

8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Baoquoc N. To whose telephone number is at 571-272- 
4041 or via e-mail BaoquocN.To@uspto.gov. The examiner can normally be reached 
on Monday-Friday: 8:00 AM - 4:30 PM, EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Breene can be reached at 571-272-4107. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is (703) 305- 
3900. 

Any response to this action should be mailed to: 
Commissioner of Patents and Trademarks 
Washington, D.C. 20231. 
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The fax numbers for the organization where this application or proceeding is 
assigned are as follow: 

(571 ) -273-8300 [Official Communication] 




